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Environment,  Management  System,  and  Operator.  Sixteen  definitions 
related  to  these  factors  were  available  to  analyze  for  reliability  and 
relationship  to  the  taxonomy  functions* 

A  survey  form  was  developed  which  included  the  above  definitions  and 
nine  selected  uric  raft  mishap  summaries.  This  instrument  was 
administered  to  a  Flight  Saiety  Officer  School  class  and  a  group  of 
naive  subjects  of  equivalent  age  and  years  of  education.  The  task  to 
be  performed  was  to  rate  Ihe  presence  or  absence  of  these  "causes" 
in  each  of  the  mishap  summaries.  Data  analysis  performed  included 
the  computation  of  mean  agreement  rates,  inter-case  consistency, 
intercorrelations  of  selected  variables,  and  comparison  of  mean 
agreement  rates  and  judged  clarity  of  the  definitions. 

/V 

The  results  indicate  that  these  factors  fit  logically  into  the  proposed 
taxonomy.  Discipline  Breakdown,  Equipment  Malfunction,  Event 
Proficiency,  and  Inadequate  Briefing  are  generally  adequate  definitions 
as  presented  in  this  study.  Channelized  Attention  and  Distraction  are 
closely  related  and  need  more  specificity  to  differentiate  them. 
Command  and  Control,  Inadequate  Training,  Lack  of  Experience, 

Lack  of  Knowledge,  Overcommitment  and  Skill /Technique  should  all 
be  revised  to  improve  their  reliability  and  clarity.  Due  to  problems 
in  developing  observable  criteria  for  judging  Mission  Stress,  Personal 
Stress,  Pressing,  and  Overmotivation,  their  current  usefulness  in 
mishap  research  is  open  to  question. 
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A  TENTATIVE  TAXONOMY  OF  HUMAN  INTERACTIVE 
FACTORS  IN  AIRCRAFT  MISHAPS 

INTRODUCTION 

Background 

According  to  Zeller  (13)  the  major  aircraft  mishap  rate  in  the  U.S. 
Air  Force  has  decreased  dramatically  since  World  War  II.  In  1943  the 
rate  per  100,  000  flying  hours  was  64.  In  1947,  when  the  Air  Force  was 
established  as  a  separate  service,  the  rate  was  down  to  44.  At  the 
end  of  the  1970's  the  rate  hovered  at  about  3  (a  reduction  factor  of  21 
since  1943),  and  has  been  near  that  level  for  approximately  a  decade. 
Additionally,  since  1947  fatal  mishaps  have  been  reduced  by  a  factor 
of  6  and  destroyed  aircraft  by  a  factor  of  7.  Both  of  these  rates  also 
appear  relatively  constant  over  the  past  ten  years,  with  little  change 
toward  improvement. 

Davis  (6)  has  suggested  that  the  major  improvement  in  reduced 
mishap  rate  has  been  in  the  hardware  area,  but  it  is  probable  that 
improved  selection  and  training  have  also  had  an  effect.  In  fact, 

Zeller  (13)  contends  that  the  human  factor  causation  aspects  have 
decreased  as  dramatically  as  material  and  other  considerations. 
Historically,  human  error  has  been  attributed  in  one-half  to  two-thirds 


of  all  miBhaps,  and  continues  at  that  level  today.  Of  these  human 
errors,  poor  technique  in  flight,  maintenance  error,  and  various 
supervisory  unsafe  acts  are  the  three  major  categories. 
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Considering  the  current  low  rate  of  mishaps  and  the  apparent 

constancy  of  human  error  involvement,  one  might  ask  whether  or 

not  the  limit  in  mishap  prevention  has  been  reached.  At  present, 

no  good  answer  to  that  question  is  available.  A  prime  reason  for  this 

is  that  approximately  25  years  of  research  in  human  factors  data 

related  to  mishaps  has  failed  to  produce  a  complete  and  cohesive 

system  for  identifying  the  second  order  factors  or  "whys"  of  human 

error  mishap  causation.  In  general,  this  failure  to  find  a  cohesive 

system  may  be  due  to  the  limitations  of  the  existent  mishap  report 

data.  In  most  studies  the  source  of  data  has  been  analysis  of  individual 

accident  reports  or  the  computer  bank  data  stored  at  the  Air  Force 

Inspection  and  Safety  Center.  The  computer  bank  data  are  elements 

translated  from  the  individual  accident  reports  by  analysts  using 

the  coding  contained  in  AFISC  Manual  127-1,  Aircraft  Accident  and 

Incident  Classification  Elements  and  Factors  (2)  and  AFISC  Manual 

127-6,  Life  Sciences  Accident  and  Incident  Classification  Elements 

and  Factors  (3).  Different  approaches  to  analysis  of  this  data  has  also 

inhibited  the  comparison  of  various  studies  for  the  purpose  of 

developing  a  cohesive  system.  However,  one  cannot  fault  researchers 

for  attempting  new  approaches.  Davis  (6)  has  succinctly  stated  the 

present  limitations  in  mishap  research  in  the  following: 

.  .  .  The  current  data  bank  of  abstracted  Air  Force 
reports  includes  only  what  might,  be  described  as 
primary  causes.  Second  level,  or  root,  causes 
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are  normally  not  included.  At  beet,  root  causes  are 
available  only  as  a  result  of  tedious  analysis  of  the  full 
reports  themselves.  .  .  .  These  are  some  of  the 
facts  of  life  which  research  and  ISD  personnel  should 
be  aware  of  when  relying  on  current  accident  reports 
and  data  banks.  .  .Although  some  feel  that  the  human 
factors  problem  cannot  be  dealt  with  any  more 
effectively  than  at  present,  unless  we  face  this  challenge 
we'll  never  know  whether  progress  in  this  area  is 
possible  or  not.  (pp.  57-58) 

The  early  research  of  the  1950's  has  been  summarized  by  Webb  and 
his  associates  (11).  Briefly,  they  found  only  about  one-third  complete 
agreement  on  underlying  causation  using  their  own  classification  system. 
Since  the  mid-1950's,  research  studies  on  human  factors  in  aircraft 
mishaps  have  tended  to  focus  on  two  general  areas:  "unsafe  acts" 
as  describes  in  AFISC  Manual  127-1,  pp.  4V-70;  and  "psychophysiological 
and  environmental  factors"  found  in  AFISC  Manual  127-6,  pp.  8-1 
through  8-4.  In  1956,  Moseley  (8)  abstracted  similiar  information 
from  individual  reports  of  2400  pilot  error  mishaps  and  found  errors 
as  the  result  of  knowledge,  training,  experience,  attention,  excessive 
motivation,  and  supervision  were  the  most  frequent  basic  causes. 

A  seeming  hiatus  of  research  studies  in  this  area  occurred  over  the 
next  15  years,  possibly  due  to  tedioumess  of  abstracting  information 
from  individual  mishap  reports  and  the  non-availability  of  computer 
bank  data  until  the  early  1970's.  During  the  1970's  Belk  (5),  Lewis  (7), 
Ricketson,  et.  al.  (9),  Santilli  (10),  and  Wegner  (12)  have  studied 
similar  data  covering  almost  a  decade  of  Air  Force  and  Army  Aviation 
mishap  reports. 
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"Unsafe  Acts"  data  (AFISC  Manual  127-1)  have  proved  to  be 
useful  as  descriptive  or  "what  happened"  information,  but  not  as 
useful  in  describing  "why  it  happened.  "  Thus  the  primary  current 
information  related  to  human  error  second  order  causes  appears 
to  be  contained  in. the  "psychophysiological  and  environmental 
factors"  (AFISC  Manual  127-6,  and  reported  by  the  mishap  boards 
on  Air  Force  Form  711  GA,  p.2).  These  factors  are  in  joint  use  by 
all  three  armed  services  in  their  reporting  systems. 

Unfortunately,  there  is  one  major  impediment  to  interpreting 
these  psychophysiological  and  environmental  factors.  The  writer 
was  unable  to  find  any  official  Air  Force  regulation  or  manual  that 
defined  the  terms  used.  Some  environmental  variables  such  as 
heat,  cold,  or  wind  blast  are  self-evident.  On  the  other  hand, 
such  psychophysiological  variables  as  "excessive  motivation  to 
suceed"  are  less  generic  and  require  some  definition  for  precise 
ju  gment.  Despite  this  limitation  a  number  of  variables  occurred 
frequently  in  the  studies  cited  above,  suggesting  their  importance 
for  future  study. 

The  problem  of  organizing  information  on  human  factors  in  mishap 
causation  have  been  addressed  by  Santilli  (10)  and  Zeller  (13).  Zeller 
has  described  a  scientific  approach  to  systematic  analysis  of  the 
design  parameters  of  both  man  and  machine  and  the  interface  variables 
between  them.  In  this  dynamic  closed  feedback  system  man's  activities 
are  a  series  of  perception/decision/response  activities.  He  suggests 
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that  a  number  of  variables  in  man's  design  should  be  examined  for 
their  potential  contribution  to  human  error.  These  are  the  physical, 
physiological,  psychological,  psychosocial  and  pathological  strengths 
or  limitations.  In  his  model  the  mishap  occurs  when  the  level  of 
demands  placed  on  man  increase  to  intersect  with  a  decreasing  level 
of  capability  to  meet  them.  Santilli  modified  thio  concept  into  an 
environmental/organism  model.  The  environment  consists  of  all 
varibles  with  which  the  organism  must  cope  (machine,  weather, 
supervision,  and  mission).  The  organism  includes  the  variables 
described  above  by  Zeller.  Both  of  these  approaches  assume  the 
operator  as  a  central  focus,  with  the  premise  that  the  analysis  of 
the  operator's  interaction  with  the  machine  and  environment  provides 
the  most  useful  information  on  mishap  causation  and  prevention. 
Purpose  and  Scope 

The  original  objectives  of  this  study  were  to  develop  a  method  of 
determining  the  relative  contribution  of  various  personnel  to  mishaps, 
and  to  determine  the  feasibility  of  developing  a  taxonomy  of  framework 
for  analysis  of  human  error  in  mishaps.  These  two  general  objectives 
remained  the  same,  but  certain  factors  intervened  to  modify  the  scope 
and  approach  used.  One  of  these  was  the  appearance  of  the  San  tilli  (10) 
and  Zeller  (13)  papers  subsequent  to  the  submission  of  the  proposal. 
These  papers  suggested  different  directions  for  the  development  of  a 
taxonomy.  Secondly,  for  reasons  outlined  in  the  background  section 
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of  this  report,  the  utilization  of  unsafe  act  data  from  the  AFISC 
computer  bank  did  not  appear  to  be  a  useful  r  oroach  to  developing 
a  taxonomy.  Essentially  the  unsafe  acts  are  at"  rather  than 
"why"  data.  This  is  not  to  deny  the  importance  of  this  data  in  a 
general  taxonomy  of  mishap  information  which  might  be  developed 
later.  However,  the  limitations  of  time  and  money  dictated  a  mod¬ 
ified  scope  and  approach.  In  collaboration  with  Dr.  Zeller  it  was 
decided  to  focus  on  second  order  causes,  their  value  as  descriptive 
terms.,  and  their  "fit"  into  a  potential  framework  or  taxonomy  of 
human  error  in  mishaps. 

The  scope  of  this  study  was  limited  in  several  ways.  First  it  was 
decided  to  consider  only  those  variables  that  had  been  found  to  occur 
frequently  in  previous  studies.  Secondly,  the  variables  selected 
would  have  to  be  adequately  defined  to  be  useful  in  this  study.  The 
only  available  definition  sources  were  the  Santilli  study  (10)  and  those 
derived  by  Lt.  Col.  John  Alberts  and  his  associates  at  AFISC. 

These  two  sources  were  considered  the  most  valid  available,  since  the 
definitions  provided  were  based  on  the  best  available  combination  of 
both  flying  experiences  and  mishap  data  analysis,  Lt.  Col.  Alberts 
also  supplied  the  mishap  cases  or  summaries  used  in  this  study, 
which  he  and  his  associates  wrote  for  Change  Pace  Analysis  (1). 

He  provided  many  more  cases  than  were  required,  and  those  used 
were  ones  which  exemplified  the  factors  studied.  In  sum,  this  study 


was  limited  tu  the  most  frequently  occurring  variables  in  previous 
rerearch,  those  with  reasonably  adequate  definitions,  those  which 
were  readily  identifiable  in  specially  developed  mishap  summaries, 
and  those  which  could  be  incorporated  into  measurement  instrument 
of  reasonable  length  and  eoi  iplexity.  Thus  a  limited  number  of 
second  order  causes  were  studied*  There  are  possibly  many  more 
that  could  be  developed  and  studied.  This  is  merely  a  first  step  in 
organizing  and  validating  an  approach  to  a  taxonomy  or  framework  for 
human  error  interaction  in  aircraft  mishaps. 

Approach 

The  approach  takei  here  was  two-fold.  The  first  purpose  was  to 
try  to  develop  a  logical  framework,  taxonomy,  or  model  for  organizing 
the  variables  to  be  studied.  The  most  recent  efforts  in  this  direction 
are  outlined  in  Santilli  (10)  and  Zeller  (13),  Some  elements  of  each  of 
these  were  included  in  a  tentative  modified  model. 

An  outline  of  a  tentative  taxonomy  of  frequently  occurring  factors  in 
aircraft  accidents  is  presented  in  Table  I.  The  table  is  divided  into 
three  major  parts,  representing  interacting  functions  in  aircraft 
mishaps:  Equipment- Physical  Environment,  Management  System, 
and  Operator.  Each  function  has  subdivisions  representing  related 
factors  within  it.  The  21  most  frequently  occurring  variables  or 
factors  in  previous  studies  are  listed  with  their  code  numbers  and 
identical  phrasing  as  they  appear  on  Air  Force  Form  711,  GA,  p.  2. 
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The  phrases  in  parentheses  next  to  each  factor  are  the  titles  of  the 
definitions  used  in  this  study  that  are  most  closely  associated  with 
the  listed  factors. 

This  framework  or  taxonomy  was  adapted  primarily  from 
Santilli  (10)  and  Zeller  (13).  Figure  1.  presents  the  relationship 
between  the  three  major  elements  of  this  concept.  The  prime 
difference  between  this  concept  and  Zeller's  is  that  the  management 
system  contains  the  primary  factors  in  the  interface  between  the 
operator  and  the  equipment  -  physical  environment.  The  major 
premise  of  this  model  is  that  the  operator  is  continuously  interacting 
with  the  equipment  and  physical  environment,  but  primarily  through 
the  medium  of  the  management  system.  Thus  the  management  system 
prepares  both  operator  and  equipment,  plans  the  interaction  of  both 
together  and  makes  decisions  about  missions  and  environmental 
elements  under  which  they  will  be  performed.  Santilli' s  (10)  approach 
deals  with  similar  variables  but  places  them  in  two  categories, 
environmental  and  operator.  Table  I  has  divided  his  environmental 
variables  into  equipment  -  physical  environmental  functions  and 
management  system  functions. 

As  can  be  seen  from  Table  I  not  all  of  the  listed  factors  were 
included  in  the  experimental  study.  The  main  reasons  for  exclusion 
were  non-availability  of  a  definition  or  appropriate  mishap  summary, 
or  limitations  in  the  length  of  the  survey  instrument  used.  Table  II 
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TABLE  I 

TENTATIVE  TAXONOMY  OF  FREQUENTLY  OCCURRING 
FACTORS  IN  AIRCRAFT  MISHAPS 


EQUIPMENT -PHYSICAL  ENVIRONMENT  FUNCTION 
Equipment 

403  Failure,  Instruments /Controls  (Equipment  Malfunction) 
Physical  Environment 

710  Visual  Restrictions-Weather,  Haze,  Darkness 
MANAGEMENT  SYSTEM  FUNCTION 
Training 

301  Inadequate  Transition  (Inadequate  Training,  Lack  of 
Experience,  Lack  of  Knowledge) 

303  Limited  Recent  Experience  (Event  Proficiency) 

304  Failure  to  use  Accepted  Procedures  (Skill /Technique) 

Supervision 

101  Inadequate  Briefing  (Inadequate  Briefing) 

102  Ordered/Led  on  Flight  Beyond  Capabilities  (Overcommitment) 

103  Poor  Crew  Coordination 

OPERATOR  FUNCTION 

Physical /Physiological 

615  Disorientation/Vertigo 

808  Inadequate  Coordination/ Timing 

Psychological/Psychosocial 

621  Inattention 

622  Channelized  Attention  (Channelized  Attention) 

623  Distraction  (Distraction) 

625  Excessive  Motivation  to  Succeed  (Overmotivation) 

626  Overconfidence  (Pressing) 

801  Habit  Interference 

807  Task  Oversaturation 

809  Misjudged  Speed  or  Distance 

810  Selected  Wrong  Course  of  Action 

811  Delay  in  Taking  Necessary  Action 

812  Violation  of  flight  Discipline  (Discipline  Breakdown) 


MANAGEMENT  SYSTEM 
INTERFACE 


CONTINUOUS  FEEDBACK  TIME  FLOW 


FIGURE  I 
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presents  the  second  order  factors  used  in  the  experimental  study. 
Definitions  of  the  16  causes  are  presented  in  the  appendix.  Three 
second  order  causes  that  do  not  appear  in  Table  I  were  selected  for 
use  in  the  experimental  study.  Command  and  Control  and  Mission 
Stress  were  found  to  be  significant  variables  by  Santilli  (10).  Their 
definitions  were  developed  by  Lt.  Col  Alberts  at  AFISC  to  describe 
causation  in  many  of  the  mishap  summaries  provided  by  him.  Personal 
Stress  is  another  concept  developed  by  Alberts,  and  appears  most 
similar  to  Preoccupation  with  Personal  Problems  (624)  on  Air  Force 
Form  711  GA,  p.2.  It  was  not  a  high  frequency  occurrence  in  other 
studies  of  Air  Force  Form  711  GA,  p.2. 

i 

The  establishment  of  the  above  taxonomy  or  framework  provides  a 
context  for  the  study  of  individual  second  order  causes.  In  the 
experimental  part  of  this  research  it  was  decided  to  focus  on  these 
basic  elements  of  the  taxonomy.  The  major  reason  for  this  is  that 
although  Santilli  and  Alberts  have  developed  authoritative  definitions 
of  these  causes,  they  are  not  yet  in  use  by  mishap  boards. 

Before  they  might  be  placed  in  widespread  use  some  evaluation  of 
their  reliability  seemed  necessary.  Across  the  various  studies  of 
causes  it  appears  that  different  researchers  have  been  using  different 
terms  to  describe  the  same  or  overlapping  concepts  of  causes. 
Reasonably  precise,  uniform  definitions  of  causes  are  necessary  if 
research  findings  are  to  produce  practical  recommendations  for 
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TABLE  II 

TENTATIVE  TAXONOMY  OF  DEFINITIONS 
USED  IN  EXPERIMENTAL  STUDY 


EQUIPMENT-PHYSICAL  ENVIRONMENT  FUNCTION 

Equipment 

Equipment  Malfunction 
MANAGEMENT  SYSTEM  FUNCTION 

Training 

Event  Proficiency 
Inadequate  Training 
Lack  of  Experience 
Lack  of  Knowledge 
Skill- Technique 

Supervision 

Command  and  Control 
Inadequate  Briefing 
O  v  e  r  c  ommitm  ent 

OPERATOR  FUNCTION 

Psychological /Psychosocial 
Channelized  Attention 
Discipline  Breakdown 
Distraction 
Mission  Stress 
O  ve  rm  otivation 
Personal  Stress 
Pressing 
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improvements  in  flight  safety.  Perhaps  more  importantly,  uniform 
definitions,  not  currently  available  to  mishap  boards,  are  a  prerequisite 
to  the  input  of  accurate  information  into  mishap  investigation  reports. 
Establishing  the  reliability  of  some  of  these  definitions  of  causes  was 
the  task  of  the  following  study. 


METHOD 

The  general  method  used  in  this  study  included  the  development 
of  an  appropriate  survey  instrument,  administering  it  to  two  samples 
and  analyzing  the  results  for  reliability  and  comparability. 

Instrumentation 

A  survey  instrument  containing  sixteen  definitions  of  causes  of 
aircraft  mishaps  to  be  applied  to  nine  mishap  summaries  was  developed 
for  this  study.  The  sixteen  causes  used  are  listed  in  Table  II  and 
defined  in  the  appendix.  lfte  instructions  for  completing  the  form 
and  an  example  of  the  cases  or  mishap  summaries  is  also  included  in 
the  appendix.  The  definitions  and  summaries  were  edited  versions  of 
those  presented  in  Santilli  (10),  Change  Pace  Analysis,  21  August  1978  (1), 
and  USAF  Destroyed  Aircraft  Analysis  (  Addendumto  Change  Pace 
Analysis,  21  August  1978)  (4). 

The  causes  used  were  generally  restri  ted  to  those  associated  with 
the  psychophysiological  and  environmental  factors  of  AF  Form  711  GA, 


iH 
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p.  2.  which  were  most  frequently  cited  in  previous  studies.  Another 
restriction  was  the  availability  of  a  reasonably  well  developed  definition 
of  the  cause.  The  definitions  found  in  the  appendix  were  composed 
of  revised  and  combined  definitions  found  in  the  studies  cited  above. 

The  editorial  changes  were  aimed  at  producing  a  briefer,  more  precise, 
and  clearer  definition.  No  claim  is  made  as  to  the  perfection  of 
these  definitions,  only  that  they  were  the  best  available  at  the  time. 

A  final  restriction  on  the  definitions  used  was  the  practical  factor 
of  length  of  the  instrument. 

The  cases  or  mishap  summaries  were  selected  using  similar 
criteria.  A  variety  of  bomber,  cargo,  fighter,  and  trainer  cases 
were  used,  with  most  being  fighters  (the  overwhelming  majority  of 
mishaps).  The  cases  were  limited  to  nine,  to  keep  the  instrument 
length  reasonable.  In  addition,  an  attempt  was  made  to  include  at 
least  two  cases  with  which  each  second  order  cause  should  be 
associated.  There  was  one  variation  in  this  selection  procedure. 

There  is  no  true  indication  of  Equipment  Malfunction  in  any  case. 
However,  there  are  two  cases  in  which  some  subject  might  see 
Equipment  Malfunction.  This  cause  was  inserted  to  determine  the 
upper  level  of  reliability  of  response  that  might  be  expected  from  the 
group  of  subjects.  It  has  been  shown  in  previous  studies  that  subjects 
are  more  consistent  in  determining  the  lack  of  presence  than  the 
presence  of  a  factor.  All  cases  were  edited  to  eliminate  or  clarify 


5 


Air  Force  abbreviations  and  jargon  for  use  with  unsophisticated 
subjects. 

The  process  of  completing  the  survey  required  the  subject  to 
study  the  definitions  and  then  determine  the  degree  to  which  each 
cause  was  definitely,  possibly,  or  not  present  in  each  case. 
Respondents  were  permitted  to  review  the  definitions  at  any  time, 
and  were  advised  that  not  all  causes  occur  in  all  cases.  At  the 
end  of  the  survey  the  subject  is  asked  to  rate  each  definition  as  very 
clear,  somewhat  clear,  or  very  vague,  and  to  comment  on  their 
usefulness. 

Subjects 

Two  groups  of  subjects  were  used  in  this  study.  One  was  a 
group  of  graduate  students  in  psychology.  There  was  no  need  or 
desire  to  sample  a  broad  population  for  the  purpose  of  this  study. 
These  subjects  were  chosen  because  they  represented  an 
unsophisticated  (technically)  group  of  an  approximate  age  and 
educational  level  of  pe  rsons  who  might  be  expected  to  perform  such 
an  analysis  in  real  life.  The  second  group  was  the  January  1980 
class  of  the  Flight  Safety  Officer  School  at  the  Air  Force  Inspection 
and  Safety  Center.  These  are  persons  who  might  be  expected  to 
perform  similar  tasks  as  members  of  a  mishap  investigation  board. 
This  sample  was  chosen  to  provide  a  group  that  would  have  some 
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sophistication  in  the  operational  areas,  if  not  in  mishap  investigation. 
The  premise  behind  selection  of  the  subjects  was  to  determine  whether 
or  not  the  definitions  of  causes  had  sufficient  clarity  of  meaning  to 
persons  of  different  levels  of  sophistication.  Ideally,  a  cause 
definition  should  be  clear  enough  to  be  commonly  understood  by 
anyone  who  might  have  official  access  to  a  mishap  report. 

Procedures 

The  instrument  was  administered  as  follows.  The  Graduate 
Students  in  Psychology  (GSP  hereafter)  were  asked  to  volunteer  to 
take  the  instrument  home  complete  it,  and  return  it  as  soon  as 
possible.  Over  a  two  month  period  27  were  returned.  Two  were 
not  included  in  the  study  because  the  respondents  failed  to  complete 
a  page  of  the  instrument.  The  Flight  Safety  Officer  School  students 
(FSO  hereafter)  were  administered  the  instrument  during  class  time 
at  the  Air  Force  Inspection  and  Safety  Center.  Twenty  six  FSO 
returns  were  received.  One  was  not  included  in  the  study  due  to 
failure  to  complete  the  last  page  of  the  instrument.  Since  anonymity 
was  assured  in  both  samples,  it  was  not  possible  to  secure  completion 
of  the  instrument  by  those  who  failed  to  do  so. 

The  subjects  were  asked  to  indicate  their  age,  sex,  and  years  of 
education,  and  their  responses  directly  on  the  instrument.  This 
information  was  transcribed  directly  on  to  optical  scan  answer  sheets 
by  the  researcher  for  data  processing.  These  answer  sheets  were 


This  data  for  the  two  samples  were  compared  for  possible 
differences.  Table  III  presents  the  means  and  standard  deviations. 

All  of  the  FSO  group  were  male,  while  64  per  cent  of  the  GSP  group 
wa3  female.  The  FSO  group  averaged  1.2  years  older  than  the 
GSP  group,  while  the  GSP  group  averaged  about  one  third  of  a  year 
more  educat.on.  F-ratios  revealed  no  significant  differences  between 
males  and  females  or  between  the  FSO  and  GSP  groups  in  age  or 
years  of  education. 

Development  of  Instrument  Key 

In  order  to  letermine  the  reliability  of  response  to  the  instrument, 
it  was  necessary  to  de  slop  a  key  to  score  the  s  object's  responses. 

Since  the  two  samples  did  not  differ  in  ago  or  years  of  education,  they 
were  combined  for  the  purpose  of  determining  their  degree  of  agreement 
on  responcey  to  each  item.  One  hundred  forty  four  items  (9  cases  time* 
16  definitions)  were  analyzed  for  direction  and  degree  of  significance  of 
agreement  (in  excess  of  change)  through  use  of  the  chi-square  statistic. 
The  first  column  of  Table  IV  lists  the  number  of  items  for  which  there 
was  a  significant  chi-square  value  at  the  .05  level  or  above. 


AGE 

YEARS  OF 
EDUCATION 
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TABLE  III 


AGE,  SEX,  AND  YEARS  OF  EDUCATION 
OF  GSP  AND  FSO  SAMPLES 


GSP(N=25)  FSO(N=25) 


MALE 

FEMALE 

.  TOTAL 

IT  S.  D. 

X 

S.D. 

IT  S.  D. 

X 

S.D. 

30.00  5.89 

29.81 

8.83 

29.88  7.89 

31.08 

7.07 

16.78  .99  ‘ 

17.  75 

.97 

17.08  1.02 

16.  72 

1.49 

i  i  nn  nsw  sr  ihjl  sssa 
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TABLE  IV 


NUMBER  OF  ITEM-  SIGNIFICANT  AT  THE  .05  LEVEL 
OR  ABOVE  ('"HI  SQUARE)  FOR  EACH  DEFINITION 


Significant 
in  Total 
Group 

Significant 
in  GSP  or 

FSO  Onlv 

Total 

Items 

Significant 

L 

Channelized  Attention 

8 

1 

9 

2. 

Command  and  Control 

6 

1 

7 

3. 

Discipline  Breakdown 

8 

1 

9 

4. 

Distraction 

7 

1 

8 

5. 

Equipment  Malfunction 

9 

0 

9 

6. 

Event  Proficiency 

7 

2 

9 

7. 

Inadequate  Briefing 

7 

1 

8 

C. 

Inadequate  Training 

6 

0 

6 

9. 

Lack  of  Experience 

9 

0 

9 

10. 

Lack  of  Knowledge 

4 

2 

6 

11. 

Mission  Stress 

7 

0 

7 

12. 

Overcommitment 

5 

0 

5 

13. 

Overmotivation 

6 

1 

7 

14. 

Personal  Stress 

9 

0 

9 

15. 

Pressing 

4 

0 

4 

16. 

Skill/Technique 

6 

1 

7 

108 

11 

119" 
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In  order  to  improve  the  validity  of  the  scoring  procedures  chi  squares 
were  computed  on  the  separate  GSP  and  FSO  group  responses  to  those 
items  that  were  not  significant  for  the  total  group*  From  the  second 
column  in  Table  IV  it  can  be  seen  that  11  more  items  were  found 
significant  using  this  approach.  This  produced  a  total  of  119  significant 
items  of  the  144  total.  The  subjects  had  the  most  difficulty  in  agreeing 
to  a  significant  degree  on  the  presence  or  absence  of  Inadequate  Training, 
Lack  of  Knowledge,  Overcommitment,  and  Pressing  across  the  cases. 
The  25  items  not  found  to  be  significant  were  keyed  "Possibly  Presont" 
for  purposes  of  further  analyses.  The  rationale  for  this  was  that 
Possibly  Present  essentially  a  neutral  response,  and  keying  in  this 
direction  would  not  significantly  increase  or  decrease  the  subsequent 
reliability  calculations. 

Agreement  Rates  and  Consistency 

One  method  of  testing  the  reliability  of  subject  responses  is  to 
determine  average  proportion  of  agreement.  This  was  done  by 
calculating  the  mean  proportion  of  agreement  of  the  total  group  to 
the  keyed  responses  for  each  definition  (across  the  nine  case;;). 

Table  V  presents  these  along  with  standard  deviations  of  proportion  of 
agreement  for  the  nine  items  of  each  definition.  The  inter-case 
consistency  (Kuder-Richardson  Formula  20)  for  the  set  of  nine  items 
representing  each  definition  are  also  indicated  in  Table  V. 
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TABLE  V 

MEAN  AGREEMENT  RATES  AND 
CONSISTENCY  OF  RESPONSE  ACROSS  CASES  (N*50) 

Mean 


1. 

Definition 

Channelized  Attention 

Agreement 

Rate 

60.  9* 

Standard 

Deviations 

11.6 

Inter-Case 
Consistency 
.  46  A 

2. 

Command  and  Control 

54.  9* 

8.2 

.37 

A 

3. 

Discipline  Breakdown 

66, 1# 

16.6 

.44 

A 

4. 

Distraction 

52.  2* 

8.3 

.02 

L 

5. 

Equipment  Malfunction 

78.  7* 

12.2 

.69 

VH 

6. 

Event  Proficiency 

58.4* 

16.0 

.27 

A 

7. 

Inadequate  Briefing 

60.1* 

16.9 

.35 

A 

8. 

Inadequate  Training 

60.  7* 

18.8 

.06 

L 

9. 

Lack  of  Experience 

70.7* 

17.9 

.01 

L 

10. 

Lack  of  Knowledge 

51.  8** 

21.8 

.15 

L 

11. 

Mission  Stress 

56.0* 

17.7 

.13 

L 

12. 

Over  c  ommitment 

43.6 

12.1 

.60 

VH 

13. 

Overmotivation 

49.  3** 

11.4 

.23 

L 

14. 

Personal  Stress 

70.0* 

9.3 

.61 

VH 

15. 

Pressing 

41.1 

17.0 

.15 

L 

16. 

Skill  /  Technique 

48.4** 

5.6 

.10 

L 

* 

** 

.  01  level  of  significance 
.05  level  of  significance 

above  chance  agreement 
above  chance  agreement 

L  Lew  to  Not  Significant 
A  Acceptable 

VH  Very  High 
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To  interpret  the  mean  agreement  rate,  it  is  important  to  remember  that 
a  33  1/3  rute  would  indicate  that  the  average  response  rate  was  at  the 
chance  level.  With  this  data,  a  47  percent  agreement  rate  is  significantly 
above  chance  at  the  .05  level,  and  52  percent  agreement  rate  i6  significant 
at  the  .01  level  of  confidence.  Thus  the  mean  agreement  rates  for 
Overcommitment  and  Pressing  are  not  significantly  above  chance,  and 
those  forLack  of  Knowledge,  Overmotivation  and  Skill/ Technique  would 
be  considered  low.  Also,  in  interpreting  the  size  of  the  mean  agreem- 
rates  several  considerations  must  be  kept  in  mind.  For  most  of  the 
definitions  with  agreement  rates  below  52,  there  was  a  strong  tendency 
for  the  respondents  to  select  Possibly  Present  across  the  nine  cases. 

The  converse  was  true  for  those  definitions  with  agreement  rates  above 
52.  This  suggests  that  a  high  level  of  agreement  also  represents  a  high 
level  of  definiteness  about  the  presence  or  absence  of  a  cause  within  the 
cases  presented. 

The  inter-case  consistency  column  in  Table  V  lists  Kuder~Richardson 
Formula  20  coefficients  across  the  nine  cases  for  each  definition.  These 
indicate  the  consistency  oi  the  subjects  in  responding  to  the  presence  or 
absence  of  a  cause  in  each  of  the  cases.  Although  there  is  no  set  standard 
for  judging  the  size  of  these  coefficients,  with  only  nine  cases  these  values 
might  be  rated  as  follows.  (It  is  unusual  to  find  KR-20  values  above  .  50  with 
nine  case,  v: 
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.50+  Very  High 

.25  -  .49  Acceptable 

.00  -  .  24  Lew  to  Non  Significant 

The  subjects  were  most  consistent  in  their  responses  to  the  presence 
or  absence  of  Equipment  Malfunction,  Overcommitment,  and  Personal 
Stress.  At  the  other  end  of  the  scale  they  lacked  consistency  across  cases 
in  responding  to  Distraction,Inadequate  Training,  Lack  of  Experience, 

Lack  of  Knowledge,  Mission  Stress,  Overmotivation,  Pressing,  and  Skill/ 
Technique.  Acceptable  levels  of  consistency  were  found  for  Channelized 
Attention,  Command  and  Control,  Discipline  Breakdown,  Event  Proficiency, 
and  Inadequate  Briefing. 

To  properly  interpret  Table  V,  it  is  important  to  use  both  the  mean 
agreement  rate  and  inter-case  consistency  in  combination  for  each 
definition.  In  general,  nean  agreement  rate  above  52  combined  with 
an  inter-case  consistency  coefficient  above  .25  suggests  a  reliable  and 
valid  definition.  Channelized  Attention,  Command  and  Control,  Discipline 
Breakdown,  Equipment  Malfunction,  Event  Proficiency,  Inadequate 
Briefing;  and  Personal  Stress  meet  these  dual  criteria.  The  other  definitions 
may  be  considered  to  have  low  to  moderate  reliability. 

The  low  inter-case  consistencies  found  for  half  of  these  definitions 
suggest  that  caution  be  used  in  interpreting  agreement  rates  as  measures 
of  reliability  in  this  and  other  studies.  The  definitions  with  low  inter - 
case  consistencies  were  generally  those  that  were  least  precise  and/or 
overlapped  with  others.  Ambiguous  definitions  are  unlikely  to  be  useful 
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even  to  well-trained,  expert  raters. 

Clarity  of  Definitions 

The  last  page  of  the  instrument  asked  the  subjects  to  estimate  the 
clarity  of  the  definitions  in  terms  of  whether  they  were  very  clear, 
somewhat  clear,  or  very  vague  .  Respondents  were  also  asked  to  comment 
on  the  usefulness  of  each  definition.  Mean  clarity  ratings  were  calculated 
for  each  scale  and  rank-ordered.  These  were  then  correlated  with  the 
rank  order  of  mean  agreement  rates  for  each  scale.  These  rankings  are 
shown  in  Table  VI.  The  correlation  of  .  722  indicates  a  high  level  of 
correspondence  between  the  respondents'  estimates  of  clarity  of  the 
definition  and  their  ability  to  detect  the  absence  of  presence  of  the  "causes" 
in  the  cases.  The  major  discrepancies  in  these  rankings  occured  for 
definitions  of  Distraction,  Inadequate  Training,  Lack  of  Experience,  and 
Pressing.  For  Distraction  and  Pressing  the  clarity  ranks  were  higher 
than  the  ranks  for  agreement  rates.  For  Inadequate  Training  and  Lack  of 
Experience,  the  opposite  was  true. 

Interrelationships  Between  Definitions 

Eleven  subjects  volunteered  37  comments  pertaining  to  all  of  the 
definitions  except  Equipment  Malfunction.  Many  of  the  comments  duplicated 
each  other  and  24  were  focussed  on  overlap  between  definitions.  Interestingly 
enough,  these  suggestions  of  relatedness  paralleled  the  tentative  taxonomy 
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TABLE  VI 

RANK  ORDER  OF  MEAN  AGREEMENT  RATES 
AND  MEAN  CLARITY. RESPONSES  (N=50) 


Channelized  Attention 

Mean 

Agreement  Rank 

Mean 

Clarity  Rank 

5 

2 

Command  and  Control 

10 

11 

Discipline  Breakdown 

4 

3.5 

Distraction 

11 

5.5 

Equipment  Malfunction 

1 

1 

Event  Proficiency 

8 

10 

Inadequate  Briefing 

7 

9 

Inadequate  Training 

6 

12 

Lack  of  Experience 

2 

5.5 

Lack  of  Knowledge 

12 

14 

Mission  Stress 

9 

8 

Ove  r  c  ommitment 

15 

16 

Overmotivation 

13 

15 

Personal  Stress 

3 

3.5 

Pressing 

16 

7 

Skill  /  Technique 

14 

13 

Rank  Order  Correlation  .  722 
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proposed  in  Tables  I  and  II  (e.  g..  Inadequate  Training  is  similar  to 
Event  Proficiency,  Lack  of  Experience,  Lack  of  Knowledge,  and  Skill/ 
Technique).  The  general  content  of  the  other  13  comments  is  incorporated 
into  the  discussion  of  conclusions  and  recommendations  of  this  report. 

Several  correlational  hypotheses  related  to  the  tentative  taxonomy 
and  the  subjects  comments  were  generated.  Under  the  Management 
System  Function  it  was  hypothesized  that: 

Ho  1.  Event  Proficiency,  Inadequate  Training,  Lack  of  Experience 
Lack  of  Knowledge,  Overcommitment,  and  Skill/  Technique 
are  significantly  intercorrelated. 

Ho  2.  Command  and  Control  and  Inadequate  Briefing  are  significantly 
correlated. 

Under  Operator  Function  it  was  hypothesized  that: 

Ho. 3.  Channelized  Attention  and  Distraction  are  significantly 
correlated. 

Ho  4.  Discipline  Breakdown  and  Pressing  are  significantly 
correlated. 

Ho  5.  Overcommitment,  Overmotivation  and  Pressing  are 
significantly  intercorrelated. 

The  21  correlations  related  to  these  hypotheses  are  shown  in  Table 
VII.  Those  between  Inadequate  Training  and  Lack  of  Experience  (.  364), 

Lack  of  Experience  and  Lack  ca  Knowledge  (.359),  and  Inadequate 
Training  and  Overcommitment  (.£94)  were  significant  at  the  .05  level, 
and  that  between  Lack  of  Knowledge  and  Overcommitment  (.369)  was 
significant  at  the  *01  level  with  the  total  group.  The  same  correlations 
were  also  computed  for  the  GSP  ai  d  FSO  samples  separately.  None  reached 


INTER  COR  RELATIONS  OF  RESPONSES  TO 
SELECTED  DEFINITIONS  (N=50) 
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significance  in  the  GSP  group.  However,  in  the  FSO  group,  the 
correlations  between  Channelized  Attention  and  Distraction  (.452), 
Inadequate  Training  and  Skill/Technique  (.415),  and  Lack  of  Knowledge 
and  Overcommitment  (.487)  were  significant  at  the  .05  level. 

These  results  tend  to  support  Ho  1  and  Ho  3  but  not  the  others. 
Inadequate  Training,  Lack  of  Experience,  Lack  of  Knowledge  and  Over¬ 
commitment  all  appear  to  fit  logically  together  under  Training  of  the 
Management  System  Function  in  the  tentative  taxonomy  (Tables  I  and 
II).  Overcommitment  was  perceived  as  more  related  to  Training  than 
Supervision.  Event  Proficiency  appears  to  be  a  more  specific  factor 
under  Training.  The  FSO  group  also  saw  Skill/ Technique  as  related 
to  Inadequate  Training.  Finally,  the  FSO  group  also  tended  to  associate 
Channelized  Attention  with  Distraction  in  the  cases.  This  suggests  that 
these  two  definitions  both  fit  under  a  general  factor  of  concentration. 

CONCLUSIONS  AND  RECOMMENDATIONS 

The  results  of  this  research  indicate  that  the  definitions 
studied  have  varying  degrees  of  reliability  and  usefulness  in 
describing  aircraft  mishap  causes.  In  the  following,  they  will  be 
discussed  in  terms  of  their  relationship  to  the  tentative  taxonomy 
proposed  in  Tables  I  and  II. 
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Equipment  -Physical  Environment  Variables 

Equipment  Malfunction  is  actually  a  first  order  factor  in  that  it 
specifies  what  happened  rather  than  why.  It  was  used  in  this  study  to 
establish  the  upper  level  of  agreement  rate  and  inter-case  consistency 
that  might  be  expected  for  the  subjects.  The  78,  7  mean  agreement 
rate  (89  in  the  FSO  group)  and  .69  inter-case  consistency  coefficient 
were  the  highest  for  any  definition,  indicating  that  it  did  serve  this 
purpose  well.  Also,  this  definition  was  ranked  as  the  most  clear. 

Recomm endation:  This  definition  could  be  improved  by  specifying 
whether  equipment  malfunction  is  operator  induced  or  due  to  other 
reasons. 

Management  System  Variables 

Eight  definitions  were  studied  under  the  category  of  management 
system  function. 

Event  Proficiency  had  a  relatively  high  mean  agreement  rate  and  an 
acceptable  level  of  inter-case  consistency.  It  is  not  significantly- 
related  to  other  training  factors.  This  definition  also  ranked  high  in 
terms  of  clarity.  Recommendation;  No  recommendation  for  improvement 
of  this  definition  is  made. 

Inadequate  Training  had  a  high  mean  agreement  rate  but  very  low 
inter-case  consistency  and  clarity  ranking.  Its  positive  correlations  with 
Lack  of  Experience,  Lack  of  Knowledge,  Overcommitment  and  Skill/ 
Technique  suggests  that  raters  consider  them  to  be  associated 
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concepts.  It  is  also  possible  that  degree  of  training  and  experience  are 
perceived  as  relatively  indistinguishable  concepts.  For  example, 
during  peacetime,  training  missions  constitute  the  majority  of  in¬ 
flight  experiences  for  crew  members  in  combat  aircraft. 
Recommendation:  Inadequate  Training  should  be  combined  with  Lack 
of  Experience.  A  minimum  standard  of  training/experience  adequate 
to  safely  perform  a  mission  element  should  be  added  to  permit  more 
precise  judgment  of  this  factor. 

Lack  of  Experience.  The  preceding  comments  about  Inadequate 
Training  fit  this  definition  also.  Indication  of  a  200  hour  minimum 
probably  influenced  the  relatively  high  clarity  rating,  but  the  inter¬ 
case  consistency  was  very  low.  One  subject  suggested  using  the 
specific  Air  Force  Command  directives  on  training /experience  to 
establish  minimal  standards  in  this  area.  Recommendations: 

This  definition  should  be  revised  to  include  a  more  appropriate 
minimum  standard  of  experience. 

Lack  of  Knowledge.  Again  this  definition  was  seen  as  related  to 
Inadequate  Training,  Lack  of  Experience  and  Overcommitment.  Both 
clarity  rank  and  inter-case  consistency  were  very  low.  Knowledge 
can  be  assumed  to  be  the  result  of  training  and  experience.  It  is  a 
management  responsibility  to  ensure  that  a  minimal  safe  level  of 
knowledge  is  attained  or  the  operator  should  not  be  permitted  to  fly 
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(except  under  immediate  supervision).  The  very  high  positive  correlation 
with  Overcommitment  suggests  these  two  definitions  are  redundant  or 
Lack  of  Knowledge  is  a  special  case  of  Overcommitment. 

R e commendation;  This  definition  should  probably  be  eliminated  or 
incorporated  as  a  specific  case  of  Overcommitment. 

Overcommitment  had  a  low  mean  agreement  rate  but  high  inter¬ 
case  consistency.  It  is  actually  a  dual  definition  as  stated  in  this  study. 

If  operators  are  assigned  to  a  task  for  which  they  are  not  prepared,  then 
the  major  factor  involved  is  supervision.  If  operators  choose  to 
perform  a  task  for  which  they  are  not  prepared  it  is  an  Operator 
Function.  The  low  mean  agreement  rate  for  this  definition  suggests 
that  the  subjects  had  difficulty  separating  the  dual  elements  of  this 
definition.  One  rater  commented  that  this  was  actually  two  definitions: 

(1)  not  prepared  for  the  mission,  and  (2)  overtaxing  of  ability /capacity. 

In  either  case  the  supervisor  should  not  assign  an  operator  to  a  mission 
unless  minimal  saf,e  performance  can  be  assumed.  Recommendation: 
Separate  definitions  for  supervisory  overcommitment  and  operator 
overcommitment  should  be  established  and  maintained. 

Skill/ Technique  had  a  low  agreement  rate  and  clarity  ranking, 
and  low  inter-case  consistency.  It  was  significantly  correlated  with 
Inadequate  Training  in  the  FSO  group,  suggesting  it  was  perceived  as  a 
sub-factor  of  training.  It  is  a  dual  definition  implying  either  a  lack  of 
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motor  skills  or  failure  to  use  proper  technique.  Recommendation. 

This  definition  should  be  eliminated  as  probably  redundant  to 
Inadequate  Training,  or  revised  as  a  sub-factor  of  training. 

Operator  Variables 

The  operator  functions  discussed  below  are  primarily 
psychological/psychosocial  factors  that  are  focused  within  the 
individual. 

Channelized  Attention  had  a  high  agreement  rate  and  clarity  rank, 
and  acceptable  inter-case  consistency.  Its  correlation  with  Distraction 
suggests  they  both  fit  under  a  general  concentration  factor.  It  is  a 
very  clear  and  specific  definition.  Recomm endation.  This  definition 
would  probably  be  more  useful  for  reporting  and  research  if  included 
along  with  other  related  definitions  such  as  Distraction  and  inattention 
under  a  general  factor  of  concentration. 

Distraction  had  a  high  clarity  ranking  and  a  good  mean  agreement 
rate,  but  low  inter-case  consistency.  Its  correlation  with  Channelized 
Attention  and  comments  by  the  subjects  suggest  both  measure  different 
elements  of  a  general  factor  of  concentration.  In  fact,  both  definitions 
imply  a  failure  to  attend  to  safety  related  cues  due  to  concentration  on 
other  stimuli.  Recommendations  are  the  same  as  for  Channelized 
Attention  above. 

Discipline  Breakdown  had  a  high  mean  agreement  rate  and 
clarity  ranking,  and  acceptable  inter-case  consistency.  It  was  not 
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significantly  correlated  with  Pressing.  It  is  a  good,  clear  definition 
in  its  present  form.  Recommendations.  None. 

Mission  Stress  had  a  reasonably  high  mean  agreement  rate,  hit 
average  clarity  ranking  and  low  inter-case  consistency.  One  subject 
suggested  that  the  effect  of  mission  stress  varies  with  the  individual, 
and  also  the  amount  of  previous  exposure  to  similar  situations.  The 
present  definition  does  not  provide  for  these  individual  differences, 
and  the  subjects  had  difficulty  agreeing  even  though  the  external 
criteria  of  Mission  Stress  were  specified  in  various  cases. 
Recommendations.  Since  it  is  difficult  to  assess  the  internal  effect 
of  Mission  Stress  on  an  individual  it  might  be  appropriate  to  simply 
record  whether  or  not  it  was  the  first  deployment,  exercise,  mission, 
or  check-ride  for  the  operator. 

Overmotivation  was  a  relatively  weak  definition  in  terms  of 
mean  agreement  rate  and  inter-case  consistency  and  clarity.  It  was 
not  significantly  correlated  with  Pressing.  Several  subjects  suggested 
that  it  might  be  synonymous  with  recklessness,  overaggressiveness, 
or  inability  to  assess  one’s  own  capabilities.  The  main  problem  with 
this  definition  is  that  it  lacks  clarity  and  specificity.  It  is  difficult  to 
determine  at  what  point  motivation  to  succeed  in  a  mission  becomes 
overmotivation.  Recommendation:  Improved  rating  of  this  factor 
requires  the  development  of  more  specific  observable  criteria  for 
judging  degree  of  motivation. 
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Personal  Stress  had  a  high  mean  agreement  rate  and  clarity 
ranking,  and  high  inter -case  consistency.  However  this  may  be  partially 
discounted  since  seven  of  the  nine  cases  had  no  indication  of  this  factor. 
Haters  are  more  reliable  in  judging  the  absence  of  a  factor  than  its 
presence.  Historically,  Personal  Stress  (preoccupation  with  personal 
problems)  has  been  reported  as  present  in  mishap  investigation  research 
in  only  about  two  percent  of  cases.  The  effects  of  Personal  Stress  vary 
with  the  individual  and  at  present  there  :1s  no  objective  way  of  measuring 
them.  Recommendation.  This  definition  could  be  used  as-is  for  purposes 
of  reporting  presence  or  absence  of  personal  stress.  However,  unless 
a  criteria  for  measuring  its  effects  can  be  developed  it  is  probably  not 
very  useful  in  mishap  research. 

Pressing  had  the  lowest  agreement  rats,  low  inter-case  consistency 
and  an  average  clarity  ranking.  It  was  not  significantly  related  to 
Overcommitment  or  Overmotivation,  *nd  was  negatively  correlated 
with  Discipline  Breakdown.  This  suggests  that  it  is  very  difficult  to 
determine  from  an  accident  report  or  summary  whether  or  not  a 
pilot  was  "taking  a  chance,"  Re  comm  endation.  Since  this  concept 
is  not  presently  measureable  for  research  purposes,  it  should  be 
revised  or  eliminated  from  the  taxonomy. 


SUMMARY 


Both  previous  research  and  the  results  of  this  study  lend  support 
to  the  tentative  taxonomy  of  human  error  interaction  outlined  in 
Tables  I  and  II.  The  intercorrelations  found  tend  to  support  the  idea 
that  Event  Proficiency,  Inadequate  Training,  Lack  of  Exp  .  t  ie  nee. 

Lack  of  Knowledge,  Skill /Technique  and  Overcommitment  are  all 
sub-factors  under  a  general  factor  of  Training.  The  high  significant 
correlations  between  Inadequate  Training,  Lack  of  Expe rietice  and  Lack 
of  Knowledge  imply  that  these  definitions  are  not  specific  enough  for 
raters  to  distinguish  between  them  well.  The  results  also  suggest  that 
Channelized  Attention  and  Distraction  are  related  sub-factors  of 
concentration  of  attention. 

The  definitions  were  analyzed  for  mean  agreement  rate,  inter - 
case  consistency,  and  clarity  ranking.  The  results  indicated  that 
Discipline  Breakdown,  Equipment  Malfunction,  Event  Proficiency, 
and  Inadequate  Briefing,  need  little  or  no  revision.  Channelled 
Attention  and  Distraction  are  related  factors  and  the  subtle  differences 
between  them  need  to  be  spelled  out  more  clearly.  Command  and 
Control  could  be  improved  by  specifying  level  and  type  cf  supervisory 
inadequacy.  Inadequate  Training,  Lack  of  Experience,  Lack  of 
Knowledge,  Overcommitment  and  Skill/ Technique  all  need  significant 
revisions  to  differentiate  them  from  each  other.  Lack  K  Knowledge 
appears  to  be  a  sub-category  of  Overcommitment,  and  Skill/ Technique 


a  sub-category  of  Inadequate  Training.  No  precise  v/ay  of  measuring 
degrees  of  Mission  Stress  or  Personal  Stress  as  they  relate  to 
mishap  causation  is  currently  available.  Until  specific  criteria  can  be 
developed,  these  definitions  are  not  particularly  useful  in  mishap 
research.  The  definitions  of  Overmotivation  and  Pressing  have  similar 
problems  of  lack  of  specific  observable  criteria  for  judging  degree 


of  motivation  and  tendency  to  "take  a  chance!'. 
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APPENDIX 


Age _  Sex _  Years  of  Education _ 

In  the  following  we  are  asking  your  cooperation  in  a  study  of  the 
definitions  of  "causes"  that  might  be  assigned  to  aircraft  mishap  cases. 

On  the  next  two  pages  are  definitions  of  a  variety  of  possible  "causes". 
These  are  followed  by  several  summary  descriptions  of  aircraft  mishaps. 

Please  do  the  following: 

1.  Study  each  of  the  definitions  until  you  feel  they  are  well  in  mind. 

2.  Rate  the  degree  to  which  you  feel  each  of  the  "causes"  exist  in 
each  summary.  Not  all  of  the  ' 'causes' '  appear  in  each  summary. 
You  are  asked  to  determine  if  they  are  DEFINITELY  PRESENT  (1), 
POSSIBLY  PRESENT  (2),  or  NOT  PRESENT  (3),  and  indicate 
your  judgements  as  to  their  degree  of  presence  after  each  "cause" 
listed  at  the  end  of  the  summary.  Feel  free  to  consult  the 
definitions  pages  at  any  time. 

3 .  After  you  have  completed  the  above,  please  enter  any  comments 
you  have  on  the  clarity  of  each  definition  and  how  it  might  be 
improved  on  the  last  page  of  this  form. 

Please  keep  the  following  in  mind.  There  are  no  specific  right  or 
wrong  answers  on  this  form.  We  are  trying  to  determine  whether  or 
not  the  definitions  and  summaries  are  sufficiently  clear  that  a  variety 
of  people  can  make  reasonable  judgements  about  them. 

We  thank  you  for  your  cooperation  on  behalf  of  the  Air  Force  Inspection 
and  Safety  Center,  the  Air  Force  Office  of  Scientific  Research,  and  the 
pilots  and  crew  members  of  the  United  States  Air  Force. 


DEFINITIONS 


1.  CHANNELIZED  ATTENTION  -  Focusing  attention  on  a  specific  task  at 
the  expense  of  ignoring  others  of  a  higher  or  more  immediate  priority. 
Channelized  attention  is  considered  a  factor  when  a  pilot  concentrates 
on  the  task  being  performed  to  the  point  that  other  cues  of  impending 
disaster  are  not  noticed. 

2.  COMMAND  AND  CONTROL  -  Command  and  control  is  considered  a 
factor  when  supervision  is  inadequate,  or  when  operating  procedures 
are  inadequate,  non-existent,  or  contradictory.  Command  and  control 
deficiencies  are  normally  characterized  by  inadequate  supervision 

at  unit  or  wing  level,  or  inadequate  mission  planning  or  scheduling. 
Failure  to  monitor  the  conduct  of  operations,  or  failure  to  provide 
close  in-flight  supervision  where  training  or  proficiency  are  suspect 
are  also  indicators  of  command  and  control  deficiencies. 

3.  DISCIPLINE  BREAKDOWN  -  Discipline  breakdown  is  evident  when 
pilots  willfully  violate  known  directives  or  restrictions.  Discipline 
is  also  considered  a  factor  when  pilots  knowingly  bend,  stretch,  or 
ignore  restrictions  because  they  perceive  tacit  approval  by  unit 
supervisors  for  the  sake  of  mission  accomplishment. 

4.  DISTRACTION  - Intenruption  of  focus  of  attention  on  a  specific 
task  by  a  non-task  related  sensory  stimulus  or  mental  process. 
Distraction  is  a  factor  when  some  unanticipated  or  unusual  event 
or  activity  interferes  with  task  performance. 

5.  EQUIPMENT  MALFUNCTION  -  The  failure  or  malfunction  of  equipment 
that  is  normally  used  in  some  phase  of  mission  completion.  This  is 

a  factor  only  when  the  failure  or  malfunction  interferes  with  successful 
accomplishment  of  a  task. 

6.  EVENT  PROFICIENCY  -  The  degree  to  which  current  training  and 
practice  were  provided.  Non-proficient  is  defined  as  :  1)  the 
operator  had  never  performed  the  task  before,  or  2)  he  had  not 
performed  it  recently  (within  eight  weeks),  or  3)  he  performed 

it  recently  for  the  first  time. 

7.  INADEQUATE  BRIEFING  -  A  premission  meeting  of  crew  members 
and  immediate  supervisors  held  for  the  purpose  of  outlining,  planning 
and  coordinating  specific  mission  objectives,  procedures  and  con¬ 
tingencies  is  a  briefing.  When  a  mission  element  that  should  have 

been  briefed  was  not,  or  was  inadequately  briefed,  briefing  is  considered 
a  factor.  Normal  tasks,  such  as  lowering  the  gear  for  landing,  are 
not  considered  mandatory  briefing  items,  and  briefings  are  not 
considered  inadequate  when  such  tasks  are  not  covered. 
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8.  INADEQUATE  TRAINING  -  Training  deficiency  is  considered  to  ve  a 
factor  when  a  pilot  has  no  prior  training,  or  was  inadequately  trained 
to  perform  the  mission  element  being  attempted, 

9.  LACK  OF  EXPERIENCE  -  Experience  is  considered  a  factor  when  a 
pilot  does  not  have  a  sufficient  background  in  the  specific  type  of 
mission  or  type  aircraft  being  flown,  or  in  the  crew  responsibilities 
being  performed.  (Less  than  200  hours  in  a  specific  role  is  considered 
lack  of  experience.) 

10.  LACK  OF  KNOWLEDGE  -  When  a  pilot  was  exposed  to  the  information 
he  needs  to  perform  the  mission  element  but  did  not  absorb  it, 

lack  of  knowledge  is  considered  a  factor.  This  assumes  no  deficiency 
in  the  training  program  provided. 

11.  MISSION  STRESS  -  If  the  conditions  surrounding  a  mission  generate 
excessive  stress,  this  is  considered  a  factor.  These  conditions 
are  often  present  during  deployments,  check  rides,  exercises,  and 
other  importajit  missions. 

12.  OVERCOMMITMENT  -  The  assignment  of  a  task  for  which  the  operator 
is  not  prepared,  or  which  in  combination  with  other  tasks,  overtaxes 
his  capacity.  When  a  pilot  chooses  to  perform  a  mission  or  is  assigned 
to  perform  a  mission  element  he  is  not  capable  of  performing,  he 

is  said  to  be  overcommitted.  Overcommittment  normally  involves 
supervisor,  schedulers,  or  flight  leaders,  but  can  also  result  from 
a  lack  of  knowledge  of  his  own  limitations  compared  to  mission  demands 
or  in  some  cases  from  a  combination  of  circumstances. 

13.  OVERMOTIVATION  -  Overmotivation  is  considered  a  factor  when  a 
pilot  is  predisposed  to  accomplish  a  given  mission  element  successfully 
regardless  of  the  situation.  Mission  success  is  afforded  a  higher 

priority  than  caution,  judgement,  or  knownrestrictions. 

14.  PERSONAL  STRESS  -  Personal  stress  may  be  a  factor  if  a  pilot 
has  unusual  or  severe  personal  problems. 

15.  PRESSING  -  A  pilot  who  continues  speeific  maneuvers  or  tasks  to  the 
point  that  known  limits  are  exceeded  is  considered  to  be  pressing  or 
"taking  a  chance.  " 

16.  SKILL/ TECHNIQUE  -  Skill  or  technique  deficiencies  are  considered 
a  factor  when  a  pilot  either  lacks  the  required  motor  skills,  or  uses 
an  improper  technique  to  perform  the  task  attempted. 
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CASE  #1 
FIGHTER  (A-7D) 


Midair  collision.  Flight  of  four  A-7's  on  a  normal  training  mission 
involving  defensive  maneuvers,  as  well  as  simulated  range  activities. 
The  briefing  called  for  number  4  to  attack  number  2,  and  number  3  to 
attack  number  1.  Aircraft  number  4  began  an  attack  on  number  2. 
Number  3  tracked  number  4,  taking  pictures,  and  during  the  picture¬ 
taking  collided  with  number  4.  The  camera  on  board  was  owned  by  the 
pilot;  it  had  been  installed  with  a  home-made  bracket,  without  the 
prior  knowledge  of  the  squadron  supervisor.  The  pilot  had  been 
advised  by  one  of  the  technicians  that  there  were  directives  which 
prohibited  using  personal  cameras  in  the  aircraft.  The  A-7  does 
not  have  a  camera  for  use  in  connection  with  air  combat  training  or 
for  recording  air-to-air  engagements.  The  camera  in  the  accident 
aircraft  blocked  the  forward  visibility  considerably  and  demanded 
constant  pilot  attention  for  operation. 


Please  circle  the  appropriate  number  after  EACH  "cause”  below. 

1.  Definitely  2,  Possible  3.  Not  Present 


4.  Distraction 


6.  Event  Proficiency 


1  2 

3 

9. 

Lack  of  Experience 

1 

2 

3 

1  2 

3 

10. 

Lack  of  Knowledge 

1 

2 

3 

1  2 

3 

11. 

Mission  Stress 

1 

2 

3 

1  2 

3 

12. 

Over  committment 

1 

2 

3 

1  2 

3 

13. 

O' v  e  rm  oti  vati  on 

1 

2 

3 

1  2 

3 

14. 

Personal  Stress 

1 

2 

3 

2 

3 

15. 

Pressing 

1 

2 

3 

2 

3 

16. 

Skill /technique 

1 

2 

3 
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Now  that  you  have  completed  the  previous  pages,  we  would  like  to 
have  your  opinion  of  the  degree  to  which  you  feel  you  understand  the 
definitions  contained  in  this  form.  Please  indicate  your  impression 
by  circling  the  appropriate  number  following  each  Term. 


1.  Very  clear  2.  Somewhat  clear  3.  Very  Vague 


In  the  space  under  each  definition  please  make  any  comments  you 
wish  about  their  usefulness. 


1.  CHANNELIZED  ATTENTION  123 


2.  COMMAND  AND  CONTROL  1  23 


3.  DISCIPLINE  BREAKDOWN  123 


4.  DISTRACTION  123 


5.  EQUIPMENT  MALFUNCTION  123 


6.  EVENT  PROFICIENCY  123 


7.  INADEQUATE  BRIEFING  12  3 


8 .  INADEQUATE  TRAINING  12  3 


9.  LACK  OF  EXPERIENCE  123 


10.  EACH  OF  KNOWLEDGE  12  3 


11  MISSION  STRESS  12  3 


12.  OVERCOMMITTMENT  123 


13.  OVERMOTIVATION  123 


14.  PERSONAL  STRESS  123 


15.  PRESSING  123 


16.  SKILL/TECHNIQUE  123 


